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ABSTRACT 


In  this  technical  report,  a  general  formula  for  the  ft-value 
of  a   2x2  matrix-differential  game  is  derived,  using  two 
different  methods. 


Prepared  by: 


1.   Introduction. 

Consider  a  two-person  zero-sum  game  with  matrix  [a..]  which  is 
being  played  for  one  unit  of  time.  If  in  a  time  interval  At  player  1 
plays  row  i  and  player  2  column  j  the  payoff  to  player  1  is  a. .At. 
Payoffs  are  accumulated  with  time  so  that  the  total  payoff  is 


a.,  dt. 

0 

Of  course,  if  instantaneous  changes  of  rows  or  columns  were  allowed  the 

integral  would  not  be  defined.   John  Danskin  [1]  has  proposed  to  resolve 

this  dilemma  by  approximating  the  game  by  n-stage  games  with  perfect 

information.   In  the  n-stage  approximation,  player  2  is  allowed  to  change 

the  column  index  only  at  times   t  =  — ;   k  =  0,1,..., n-1,   while  player  1 

n 

k   a 
is  allowed  to  change  the  row  index  only  when   t  =  --  +  — ,   where  0  ^  a  £  1 

n   n 

is  a  fixed  parameter.   The  payoff  is  then 

1 


a.,  dt  =  —  J  aa.    .  +  (l-a)a.  .  . 


The  row  index  i„   is  called  the  predecessor.   We  denote  this  game 

G  (cr;i_)   and  call  it  the  n-stage  ft-game  with  predecessor   i  .   Its  value 
n    0  u 

1  n 
ft  (a;i_)  =  min  max  ...  min  max  —  )      aa.         .     +    (l-a)a.  .      (1.1) 

i.   i,      ji    k=l   lc-1  k  k  k 

Jl   1      Jn   n 

John  Danskin  has  proved  (in  a  far  more  general  context)  that  the  sequence 

JJ1(a;i0),n2(a;i()),...  (1.2) 

converges  to  a  limit  which  does  not  depend  on  the  predecessor   i_.   This 
limit 


JJ(o)  =  lim  fi  (a;irt) 
n    U 

is  then  called  the  fl-value  of  a  matrix-differential  game   [a..]. 

ij 

The  main  purpose  of  this  technical  report  is  to  establish  the 
following  formula  for  the  ft-value  of  a  two-by-two  game. 


Let  A  = 


all   a21 


L_a12   a22J 


be  a  matrix,  let   I  =  {1,2}   be  the  index  set,   let 


0  £  a  £  1 


Denote  M  =  max  max  {a. . } ,   M  =  min  min  {a. . } , 
1€I  j€I   1J       1€I  j€I   1J 

V  =  min  max  { a .  .  }  ,   V_  =  max  min  { a .  .  }  ,   and 


Then 


jGI    i€I 


ij 


1€I  j€I 


ij 


v(a)    =  -[a  (MHO   +   (l-a)(M+V)]. 


fi 


V     if     v(a)    ^  V, 

)      if     V  <   v(a)    <   V, 


(a)    =       v(a 

V     if     v(a)    ^  V. 


(1.3) 


We  provide  two  independent  proofs  of  this  formula,  each  using  a 
different  approach — Shubert's  (Sec.  2)  or  Washburn's  (Sec.  3).   These  two 
sections  can  be  read  independently.   The  notation  is  common.   A  brief 
comparison  and  discussion  of  the  two  approaches  is  contained  in  Sec.  4, 
together  with  some  ideas  about  possible  generalization  to  larger  games. 


2.   Shubert's  Approach. 

From  (1.1)  it  is  easily  seen  that  ft  (a;*)   satisfy  the  recurrence 
relation 

Q  ,,  (cr;k)  =  min  max{— —  [aa.  .  +  (l-o)a.  .]  +  -§r  n  (cr;i)>,   k  6  I,    (2.1) 
n-r±  n-rl    Ki         ii     n+1  n 

j€I  i€I        J         J 

with  initial  condition  fi„  =  0.   This  relation  could  be  used  to  evaluate 

ft (a) ,   which  can  be  done  on  a  computer  provided  numerical  values  for  a 

and  the  entries   a. .   are  given. 

However,  if  we  wish  to  obtain  the  entire  function  ft(*)   and  establish 

some  of  its  properties,  the  recurrence  relation  (2.1)  may  be  of  little  help. 

An  analogy  with  similar  problems  suggests  that  perhaps  ft(*)   can  also  be 

found  as  a  solution  of  a  functional  equation  obtained  from  (2.2)  by  letting 

n  ->  °°.   Unfortunately,  such  a  process  would  yield  the  equation 


fl(»  ;k)  =  min  max  ft(*;i), 
j   i 

which  is  satisfied  by  any  function  such  that  ft(a;k)  =  ft(a;£)   for  k  ^  I. 
In  what  follows  we  are  going  to  show  that  the  difficulty  can  never- 
theless  be  circumvented  by  solving  the  equation 

Q(cr;k)  =  min  max  {A  [aa  +(l-a)a.  .  ]  +  (l-A)fl(o;i) } ,   k  €  I,    (2.2) 
j€I  i€I      J       1J 

where  0  <  X   £  1   is  a  parameter. 

We  first  prove  that  this  equation  has  a  unique  solution,  depending, 
of  course,  on  the  parameter  \.      Next  we  show  that  as  \   ->  0   this  solution 
converges  to  the  fi-value.   As  a  byproduct  we  obtain  a  new  proof  for  the 
convergence  of  the  sequence  (1.2).   Finally,  by  actually  finding  the  solu- 
tion of  (2.2)  for  small   A  >  0  and  letting  A  ->  0  we  establish  the  formula 
(1.3)  for  the  fi-value. 


We  begin  with  some  definitions.   To  simplify  notation  we  temporarily 
suppress  a   and  abbreviate 

h.  .  =  era,  .  +  (l-a)a.  .  , 

where   i  (;  I ,   j  G  I ,   k  6  I . 

Let  x  =    (x..  ,x9)  6  Re  be  a  two  component  vector,  let  0  £  X  £  1. 
Denoting 

Hk.(X,x)  =  Xhk.  +  (l-X)x. 

16  1,  k  €  I,  j  €  I, 

we  define  a  parametric  family  of  mappings 


T(X,x)  =  (T1(X;x),T2(X,x)) 

by  (2.3) 

T  (X ,x)  =  min  max  H . . (X ,x) ;   k  C  I. 
k       j€I  i€I  1J 

Let  2ii 'iio  >  •  •  •   be  a  sequence  of  vectors  from  Re2  defined  by 

xx  6  Re2,  2^  "  T(^,£,),   n  =  l,2,...  (2.4) 


Comparing  with  (2.1)  we  see  that  with  x     =  0_ 

xn+1  =  (fin(a;l),nn(a;2)), 

so  that   x  ,x9 , . . .   is  indeed  the  sequence  defining  the  ft-value  of  the 
fi-game  with  matrix  A. 

Next,  let    |*|    be  the  maximum  norm  on  Re2 


|xj  |  =  max{ |x  j , |x  | }, 


let   a  be  a  constant  such  that 


max  max  | a . . |  £  a . 
i€I  j€I   1J 


Let 


X  =  {x  €  Re2:   |x|   £  a} 
We  are  now  ready  to  state  the  following  theorem 


Theorem  1 :   Let  0  ^  X  £  1,   let  T  be  the  mapping  defined  by  (2.3)  and 
restricted  to  the  domain  X,   let  x.i  »2E.o » *  * "   be  the  sequence  (2.4).   Then 
the  following  is  true. 

(1)  For  every  0  <  X  £  1.   there  is  a  unique  fixed  point 

x(X)  =  T(X,x(X)). 

(2)  The  limit 


lim  x(X)  =  £  =  (£,,£,) 

X+0 

exists  and   £..  =  £~. 


(3)   The  sequence  x.  ,x~»...   converges  and 


lim  x  =  lim  x(X) . 
ir*10      x->0 


Proof :   The  proof  is  divided  into  a  sequence  of  eight  lemmas.   Lemma  3 
proves  statement  (1) ,  Lemma  7  statement  (2)  and  Lemma  8  statement  (3) . 
We  would  like  to  point  out  that  the  proof  may  seem  rather  uneconomical 
for  the  two-by-two  case  (for  instance,  Lemma  3  can  be  proved  without 
using  Banach's  theorem) . 

We,  nevertheless,  leave  it  in  its  present  form  with  an  eye  on 
possible  generalization. 

Lemma  1 :   For  any  0  £  A  £  1 

x  €  X  =>  T(X,x)  €  X 
Proof : 

|  |T(X,x)|  I  =  max{|T  (X,x)|  ,  |T2(X,x)|} 

^  max  max  max  |Xh. .  +  (l-X)x. | 
kei  i€I  j€I    1J         X 

£  Xa  +  (1-X) | |x| |  £  a   if  x  £  X. 

Lemma  2 :   For  any  0  ^  X  £  1 

||T(X,x)  -T(X,x')||  <;  (1-X)  |  |x-x!  |  | 

Proof:   If   (f  ,f  )   and   (g  ,g  )   are  any  two  real  vectors  then 

|max{f1}f2}  -  max{g1,g2)  £  max{  |  f^g^  ,  |f2-g2|  }, 
and 

|min{f  ,f2>  -min{g1,g2}|  £  max{ | f 1~g1 | , | f 2~g2 | } . 

Using  these  two  inequalities  in  the  reversed  order  we  obtain 


i  till  k  k 

|T(X,x)    -T(X,_x)|       =  max|min  max  H.  .  (X  ,x_)    -  min  max  H  .  .  (X  ,x' ) 

k€I   j€I   161  1J  j€I   161      1J 

i      k  k 

g.   max  max  |  max  H..(X,x)  -   max  H .  .  (X  ,3c*  )  | 

k€I  J6I  161  LJ       161  XJ 

I  k  k       i 

£  max  max  max | H . . (X ,x)  -  H . . (X ,x ' ) | 

k6I  j6I  161  XJ         rj 


=  max  I  (l-X)(x.-x'.)|  =  (1-X)  |  |x  -  x*  |  |  . 
i6I         1  X 


Lemma  3 :   For  every  0  <  X  £  1   there  is  a  unique  x  6  X  such  that 
T(X,x)  =  x\ 

Proof :   By  Lemma  2  the  map  T   is  for  any  fixed  0  <  X  £  1  a  contraction 
and  by  Lemma  1  it  maps  a  closed  bounded  set  X   into  itself.   Hence  by 
Banach's  fixed  point  theorem  it  must  have  a  unique  fixed  point. 

Lemma  4 :   For  every  0  <  X  ^  1   there  is   p  6  I   and   q  6  I   such  that 
either 

xx(X)  =  h|   ,  x2(X)  =  h]_  +  *(h£q-hlp),  (2.5) 


or 


xl(X)  -h|q  ,?2(X)  =h|q+X(hlp-h2q),  (2.6) 


or 


hj  +  h^   -  Xh? 

r  (x\  =    2p       iq        iq 

1U)        2  -  X 


hi   +  h*   -  Xhl 
x2(X)  -   2P   2  \ ^  , 


(2.7) 


or 


xx(X)  =  h|   ,  x2(X)  =  h*  .  (2.8) 


fs*t  tf+* 


Proof :   Since  x  =  (x  ,x  )   is  a  fixed  point  of  the  map  T(A,*)   we  must 

have 

x,  =  h*   +  (l-X)x. , 
1     lp        i' 

x2=  hjq+  (l-X)x., 

where   i  €  I ,   j  €  I,   P  €  I>   q€I.   Choosing  values   1  or   2   for  the 
index  i  and  the  index  j   we  obtain  four  sets  of  pairs  of  linear  equations 

x,  =  Ah}   +  (l-X)x. 
1     lp         1 

x2  =  Xh*  +  (1-X)^ 


x±   =  Xh*  +  (l-X)x, 


x2  =  Xh*  +  (1-X)x2 


-i  -  *h2p  +  ^-^2 


x2  -  Xh^q  +  (1-X)^ 


x,  =  Xh}   +  (l-X)x. 

i     lp         1 

Z,  =  xhfq  +  (i-x)Z2 

The  solution  of  each  set  gives  the  corresponding  expression  (2. 5) -(2. 8) 


Lemma  5 ;     lim  |x  (X)  -  x  (X) |  =  0. 
X+0 

Proof:   Assume  the  statement  is  false,  i.e.  there  is   e^  >  0   such  that 

for  every  X      >   0   there  is   0  <  X  <  X    for  which 

Ix^CX)  -  x2(X)  |  :>  eQ. 


By  Lemma  4  this  could  only  be  if  2i(X)   is  given  by  (2.8)  i.e. 


and 


x2(A)  =  hlp  ,  x2(X)  =  h^  (2.9) 


hlp  -  h2ql  *  V  (2'10) 


Next  since  x(X)   is  a  fixed  point 


Hi  (A,x)  ^  Hi  (X,x), 


and 


x  (X)  =  min  max  H .  .  (X  ,x)  ,    k  U, 
j€I  i€I  1J 

and  because  of  (2.9)  this  implies 


Substituting  from  (2.9)  we  obtain 

Xh!   +  (l-X)hl   £  Xhl   +  (l-X)h*  , 
lp         lp     2p         2q 

and 

Xh2  +  (l-X)h?   ^  Xh2  +  (l-X)hl  , 
2q         2q     lq         lp 

which  is  equivalent  to 

hi  -  h2  <;  X(h.i  -h§  ). 
lp     2q       lp   2q 

However,  these  two  inequalities  cannot  hold  for  arbitrarily  small  X  >  0 

unless  h}   =  h£  ,   which  contradicts  (2.10).   The  lemma  is  proved, 
lp    2q' 


10 


Lemma  6:   Let   f_  =  (f   , . .  .  >f.)   and   e   =  (g   , . . .  ,g  ) ,   j  =  l,...,n 
be  given  vectors.   Let  for  every  0  <  X  £  X   at  least  one  of  the  n 


inequalities  Xf_.    £  j>   be  satisfied, 


Then  there  is  a  X   6  (0,A  ]   such  that  if  the  inequality 

Xf.   £  g.    is  satisfied  for  A  =  A^,   it  is  satisfied  for  any  0  <  X  £  X^. 
-^o   "%  °  ° 

Proof:   Since  all  these  inequalities  are  linear  there  is,  for  every  j, 

a  closed  interval  J.  c  [0,X  ]   such  that  the  j-th   inequality  is  satisfied 

if  and  only  if   X  ^  J . .   Although  some  of  those  intervals  may  be  vacuous 

the  system  {J  ,...,J  }  must  cover   (0,X  ]   since  by  assumption  at  least 

one  of  the  inequalities  is  satisfied  for  any  X  £  (0,X,].   Hence,  there 

must  be  at  least  one  interval  with  nonempty  interior  and  left  endpoint 

zero,  say  J.   =  [0,X_],   Xn  >  0,   and  the  statement  follows. 
3Q  U     U 

Lemma  7 :   There  exists   Xn  >  0,   a  vector  jj_  =  (£  ,£  ) ,   and  a  vector 
b_  =  (b  ,b  )   such  that  either 

x(X)  =  I  +  Xb   for  all  0  <  X  £  A  ,  (2.11) 

or 

x(A)  =  ^2+_XY  for  all  0  <  A  £  A  .  (2.12) 

In  either  case  the  vector  £  is  such  that 


Proof:   By  Lemma  5  the  fixed  point  x^(A)   is  given  by  one  of  the  expressions 
(2.5)-(2.8).   Since  p  and   q  generally  depend  on  A   this  leaves   16 
possibilities. 


11 


Since  x(\)      i-s  a  fixed  point  there  is   p  £  I,   q  6  I,   r  £  I, 
s  £  I,   in  general  depending  on  A,   such  that 

xn  =  H1  (X  ,x)  =  min  max  H*  .  (X  ,x")  , 

1      pq  ja  i€i    1J 

and 

x     =  H2  (X,x)  =  min  max  H2.(X,x). 
1         rs       j€I  i€I  1J 

But  A  =  min[max(A,B) ,  max(C,D)]   is  equivalent  to 

A  ;>  B  and  A  £  C 
or 

A  ^  B  and  A  <;  D. 

Hence,  (2.13)  is  satisfied  (with  some  specific  p,  q,  r,  s)   if  and  only 
if  at  least  one  of  the  resulting  four  systems  of  four  inequalities  between 
pairs  of  the  H's   is  satisfied.   Now,  each  of  these  inequalities  is  of 
the  form 

Xh  +  (l-X)x^Xh'  +  (l-X)x' ,  (2.14) 

where  h  and  h   each  stands  for  one  of  the  h..  s  and  x  and  x 
each  for  one  of  the  two  components  of  x. 

By  Lemma  6  there  must  be  X   >  0   such  that  for  0  <  X  £  X    x(^) 
cannot  be  given  by  (2.8)  unless  hj   =  h2  . 

Hence,  substituting  for  x  and  x'   any  one  of  the  expressions 

(2.5)-(2.7)  or  the  expression  (2.8)  with  hj   -  h2    the  inequality  (2.14) 

lp    2q 

becomes  either 

X(h-h')  +  (l-X)XB  ;>  0 
or 

(2-X)X(h-h')  +  (l-X)XB  >   0 


12 


where  B   is  a  constant  involving  the  h   's.   But   X,   being  positive, 
can  be  canceled  and  the  inequalities  become  linear  in  X. 

Thus  we  have  a  finite  number  of  systems  of  four  linear  inequalities 
and  since  the  fixed  point  _x  exists  by  Lemma  3  for  every  0  <  X  £  1   at 
least  one  of  the  systems  must  be  satisfied  for  every  0  <  X  _£  X  .   Hence, 
by  Lemma  6  there  is  a  X   >  0   such  that  if  a  particular  system  is  satis- 
fied for  X  =  X„   it  remains  satisfied  for  all   0  <  X  ^  X_.   Consequently, 
the  indexes   p,  q,  r,  s,   which  identify  the  fixed  point  for  X  =  X_ 
will  remain  unchanged  as   X   decreases  to  zero.   Hence  _x(X)   will  for 
every  0  <  X  £  X   be  given  by  one  of  the  expressions  (2. 5) -(2. 7)  with  p 
and  q  no  longer  changing  with  X   and  thus  by  Lemma  5  must  have  one  of 
two  forms  (2.11)  or  (2.12)  with  E,     =   E,    .      The  lemma  is  proved. 

Lemma  8:   Let  x  ,  x_  , .  .  .   be  a  sequence  satisfying  (2.4),   let 

£  =  lim  x(X) .   Then  the  sequence  x. ,x.,...   converges  and  lim  x  =  £ . 
—   ,  rt  —  — 1  —I  — n   — 

X->0  n-x» 

Proof :   Let  e   >   0,   let  N  be  a  positive  integer  such  that 

1 
and 


NS10  (2-15) 


1      n      1 

i^J.i+k^K  for  every  n  =  0,l,...,  (2.16) 

k=0 

—2 

where  K  =  max{  |  |b_|  |  ,  |  |5_-2bJ  |  (2-Xn)   }   and  ^njb_  are  those  of  Lemma  7. 

Such  an  N  exists  for  any   e  >  0   since  the  harmonic  series  is  Cesaro 
summable  to  zero. 

Let  n  be  a  nonnegative  integer.   By  Lemma  2 

I  I  x:    -  x  (—)  II  =  I  I  T  (—  x     )  -  T  f—  x  (—)  I  I 
1  l2%+n   -^N+n;  '  '    I  I  UN+n  '%+n-r    UN+n  '-^N+n;  '  ' 

N+n-1 1|         *»,   1  N  i  i  /o  i  -7\ 


13 


and  by  triangular  inequality 

~/       1      s    I     I  II  ~/ 


i=wi  -  *W  1 1 £  I  liw-i  -  *<s*h:>  1 1 + 1  life)  -  *<5sb:)  1 1    (2-18) 


By  Lemma  7  if   0  <  X  £  Xn  and  0  <  X '  £  X„ 

|  |jT(A)  -  x(X')|  |  <;  |x  -  X'|  |  |b|  | 
or 

||x(X)  -  x(X')||  <;  |X  -  X'|  |^x")^^ 

depending  on  which  of  the  two  expressions  (2.11)  and  (2.12)  applies. 
Hence 

I  l-(N+n")  "  -(N+n-l}  '  '  *  (N+n)  (N+n-1)  (2.19) 

Putting  (2.17),  (2.18)  and  (2.19)  together  and  calling   11*™+  -  Efer-)  I  I 


=  A   we  have 
n 


N+n-1        K_ 

n  *  N+n   n-1    (N+n) z 


Iterating  we  obtain  eventually 

+  -5-  V  i 

^0   N+n  ,  £  k 


n   N+n  C 

k=0 


for  every  n  =  0,1,...   and  hence  by  (2.15)  and  (2.16)  and  the  obvious 

fact  that   0  £   |A    £  a,   the  radius  of  X,   we  have 

i  i  ni  i 

■ 

A  <  e   for  every  n  =  0,1,...  . 
n 

Thus   lim  A  <  e   for  any  e  >  0   so  that   lim   |x  -  x(— )    =  0  and 

n  '  '— n   —  n  '  ' 

n-H»  n-x» 

since  by  Lemma  7   lim  x(— )  =  E   the  proof  of  Lemma  8  and  also  of  Theorem 

_  n        J2. 

n-*°° 
1  is  terminated. 


14 


Now  we  can  proceed  with  the  proof  of  the  formula  for  the  ft-value, 


Theorem  2:   The  formula  (1.3)  is  true. 


Proof :   Since  always  V  £  V  we  distinguish  two  cases. 

Case  1:  V  <  V 

With  no  loss  of  generality  assume  that  the  matrix  A   is  such  that 


all  "*  a22  ^  max{a12,a21>, 


relabelling  the  entries  if  necessary.   Then  we  have 


M  =  a   ,   V  =  a    and  V  =  max{a  „,a  J. 


We  will  again  use  the  notation 


hk.  =  aa.  .  +  (l-a)a..   and  H^.  (X,x)  =  Xhk.  +  (l-X)x.  . 
ij     kj         ij        xj   '-      ij         k 


Notice  that  in  this  notation 


v(o)  -  ICh^+hf,) 


and  that  if  k=i,   h..  -   a. .. 

il     ij 


Case  1.1:  v(a)  ^  a 

Let 


*1  =  *h22  +  (1"X)h22 


x„  =  h? 


(2.20) 


'2    22 
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so  that 

H^2(X,x)  =  x1     and  H|2(X,x)  =  x2  (2.21) 


Next 


H22(A,x)  -  Hl2(X,x)  =  X(h22-hj-2)  +  (l-X)(x2-x1) 


=  X[h22  -  h|2  +  (1-X) (h|2-hl 2) ] , 

which  is  nonnegative  in  a  positive  neighborhood  of  zero  if  and  only  if 
h22  "  h12  ^  ^'   But   h22  =  a22'   ^12  =  al2   SO  that  indeed 

H22(X,x)  -  H|  (A,x)  :>  0   for  small  X  >  0. 


Next 


H22(X,x)  -  H^U.x)  =  X[h22  -  h^  +  (l-X)(h22-h22)]  <;  0 

if  and  only  if  h^  £  hj  ,   which  is  true  since  h^   =  a    and  h*   =  a 
Thus  for  small   X  >  0 


H*  (X,x)  =  min  max  Hj  (X,x)  (2.22) 

J    i   1J 

Similarly 

H22  "  HU  ^  °  °  2h22  "  (h12+h22)  *  °  "  2a22  *  a22  +  a12' 


and 


H22  "  Hll  *  °  *  2h22  "  2v(a)  *  °' 


which  is  true  since  h^   =  a~_.   Hence,  for  small  X  >  0 


H^  (X  ,x_)  =  min  max  H^  .  (X  ,x)  » 
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which  together  with  (2.21)  and  (2.22)  means  that  for  all   X  >   0,  in  some 
neighborhood  of  zero  (2.20)  is  a  fixed  point  of  the  mapping  (2.3).   Conse- 
quently by  Theorem  1  the  fi-value 

0(0)  -  lim  x.  -  h*   =  a   . 

x->-o 


Case  1.2:  v(a)  ^  V 


Let   p  €  I   and   q  €  I   be  such  that  V  =  a   .   Notice  that   p  f   q 


so  that 


v(a)  =  hh9     +  hq  ]. 
2   qq     pp 


Let 


so  that 


x  =  hP   ,   x  =  Xhq  +  (l-X)hP  (2.23) 

p   pq    q    pp      pq 


x  =  HP  (X,x)   and  x  =  Hq  (X,x).  (2.24) 


pq   -        q    PP 
Similarly  as  in  Case  1.1. 


HF   -  Hp  ^0  «  2hp   -  2v(a)  >   0, 

pq   qq        pq 


which  is  true  since  h   =  a   =  V.   Next 


pq   pq 


rP  -  .  /i,P  _t.P 


so  that 


HF   -  HF   =  X(hF  -hK  )  =  X(a   -a   )  £  0, 

pq   pp    pq  pp     pq  pp 


HP  =  min  max  Hp.  .  (2.25) 

pq    j   ±   U 


Similarly 
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since 


and 


sxnce 


Hq  -  Hq  :>  0  o  hP  -  hq  ;>  0 

pp   qp        pq   qp 


hP=a   =  V  ^  M  =  a   =  hq  , 

pq   pq  -      qp   qp 


Hq  -  Hq  £  0  o  hp  -  hq  £  0 

pp   qq       pq   qq 


a   £  a     Hence 

pq   qq 

Hq  =  min  max  Hq .  (2.26) 

PP     .    .    ij 
J    l 

=  v,P  = 


and  (2.23)-(2.26)  imply  fl(a)  =  hp  =  V. 


Case  1.3.  V  <  v(a)  <  V 


Calling  again  _V  =  a    we  now  take 


x  -  -±-   (2v(a)  -  Ahq  )  , 
P    2-A  pp 


x  -  ^-   (2v(a)  -  AhP  ) 
q    2-A  qq 


We  have  now 


Next 


so  that  for  small   A  >  0 


HP   -  HP  ;>  0   if  and  only  if   2(hP  -hP  )  +  hq  -  hP  £  0 

qq   pq  qq  pq    pp   qq 


But  this  is  equivalent  to 


2v(a)  -  2h^  ^  0, 

pq 


(2.27) 


x  =  HP  (A,x)  ,  x  =  Hq  (A,x).  (2.28) 

p   qq  -    q   pp  - 


HP   -  HP   =  A[hP   -  hP  +^4  (^  "hP  )] 

qq   pq     qq   pq   2-A   pp  qq 
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which  is  true  since  h   =  a   =  V.   Similarly 

pq    pq   - 

HP   -  HP  £  0  «  2(hP  -hP  )  +  hq   -  hP  £  0  «  2v(a)  -  2hP  £  0 
qq     pp  qq   pp      pp     qq  pp 

since 

hP  =  a   ^  V  and  v(a)  <  V. 
PP    PP 


Thus 


HP  =  min  max  Hp .  (2.29) 

qq    .   .   ij 

J    i 


Finally 


Hq   -  Hq  ;>  0  «  2(hq  -hq  )  +  hP  -  hq  ;>  0  «  2v(a)  -  2a   :>  0 

pp   qp        pp  qp    qq   pp  qp 

since  a   =  M,   and 

qp  - 

Hq  -  Hq  <;  0  «  2(hq  -hq  )  +  hP  -  hq  :£  0  «  2v(a)  -  2hq  <;  0 

pp   qq        pp  qq    qq   pp  qq 

since  v(a)  <  V  ^  a      .   Hence 

qq 

Hq  =  min  max  Hq. ,  (2.30) 

PP     •    •    ij 
J    i 

and  (2. 27) -(2. 30)  yield 

ft  (a)  =  v(a)  . 

Case  2:  V  =  V 

Although  we  could  use  the  same  procedure  as  in  Case  1  again  we  prefer 
a  simpler  argument.   If  V  =  V  =  V   then  the  matrix  A  has  a  saddle,  say 

a,   =  V.   Since  every  n-stage  ft-game  G  (a;k),   being  a  game  with  perfect 

tcJc  n 

information,  has  a  pure  value  we  must  have 


ft  (a;k)  =  V   for  every  n  =  1,2,... 
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But  by  Theorem  1   lim  ft  (a;i)  =  ft (a)   regardless  of   i  £  I.   Hence 

n 

ft (a)  =  V   for  all  0  ^  o   £  1  and  the  theorem  is  proved. 
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3.   Washburn's  Approach. 

We  will  proceed  to  prove  formula  (1.3)  by  showing  that  Player  I 
can  get  at  least  ft  (a)  +  0(l/n) ,   that  Player  II  can  prevent  player  I 
from  getting  more  than  ft   (a)  +  0(l/n) ,   and  that  ft  (a)  =  ft   (a)  = 
ft (a) .   The  two  functions  ft  (a)   and  ft   (a)   will  be  defined  within  the 
theorem.   For  future  reference,  let  the  clockwise  and  counterclockwise 
averages  be 

v  (a)  =  .5((ai;L+a  2)(l-a)  +  (a12+a21)a), 


and 


v   (a)  =  .5((a  +a_.)a  +  (a. _+a_1) (1-a)) . 
cc  11  11  11      21 


Theorem:   Formula  (1.3)  is  true. 


Proof:   We  will  show  that  players  I  and  II  can  asymptotically  confine  them- 
selves to  four  strategies  each,  those  strategies  being  the  four  functions 
mapping   {1,2}   onto  itself.   For  example,  the  strategy   (2,1)   for  I 
means  that  I  will  always  choose   2   if  II  has  previously  chosen  1,  or 
1   if  II  has  previously  chosen  2.   We  will  use  the  symbol   s  ■+■   x   to  mean 
that  the  strategy  s   for  I  assures  that  the  average  payoff  to  I  will  be 
at  least  x  +  0(l/n) ,   where  n  is  the  number  of  moves,  regardless  of 
the  starting  state.   Similarly  for  II. 
We  proceed  by  showing  that 
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(1,2)   -*  min{vc(a)  ,a1;L,a22} 
(2,1)   +  min{vcc(a) ,a12,a21} 

(1.1)  +  min{a1;L,a12} 

(2.2)  ^  min{a21,a22} 

(1,2)    fl  max{vcc(a),an,a22} 
(2,1)    ^  max{vc(a) ,a12,a21> 

(1.1)  ^  max{an,a21} 

(2.2)  |^.   max{a12,a22>  . 

For  example,  consider  the  strategy   (1,2)   for  I.   After  n  stages,  let 
n..(n. .)   be  the  number  of  visits  to  state   (i,j)   initiated  by  I (II). 
Then  as  long  as  I  uses   (1,2),   regardless  of  the  starting  point, 

I     II    II   1 

nll  *  n21  +  nll  "  1 

I     II    II 
n22  *  n^  +  n22  -  1 

l  II    Hi 
|nn  -  n12|  £  1 

n21  =  ° 

I   I   nii  =  n  • 

i  J 

Since  the  average  payoff  to  I  is 

I     _  1  v  r  .   I  f.      N     II   v 

Vd,2)  -nU  (nij(1-ff)  +  nij  a)air 
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a  lower  bound  on  v,    .   can  be  obtained  by  solving  a  linear  programming 
problem.   Furthermore,  if  we  define 

I   _  I  ,  ,        II    II, 

p..  =  n../n  and  p..  =  n../n, 
ij     iJ  ij     ij 

and  neglect  terms  that  are  0(l/n),   the  program  is 

min  I   liv\.{l-a)   +   p**  a]a..  i  v(1>2) 

K-     *.   4.  I  II        II 

subject  to  Pll  =  p21  +  Pll 

I  II    II 
P22  =  P12  +  P22 

II  II 

P21  =  P12 

1      1  n 

P21  =  P12  =  ° 

n  pJJ  - 1  '; 

There  are  six  equations  in  eight  variables.   There  is  an  optimal  solution 

in  which  at  least  two  of  the  variables,  in  addition  to   p    and  P -,.->> 

-rr  u  ,  vi         II        II    u     I     II 

are  0.   If  these  two  variables  are  p    and  P99>   then  p   =  p   = 

P12  =  P22  =  *5   and  V(l,2)   is  Vc(a)'   If   Pll  =  °'   V(l,2)  =  a22' 

and  if  p   =  0,   v.    .  =  a   .   If   p2_  =  0,   v,    .   is  some  mixture  of 

a.. ,   and  a__.   In  all  cases,   (1,2)  ■+   min{v  (a)  ,a, .  ,a__}  .   Essentially, 
II         IJ.  1       C      11  2.2. 

II  has  his  choice  of  either  going  around  in  a  clockwise  circle  or  else 
acepting  one  of  the  two  payoffs  that  I  keeps  trying  to  obtain.   The  rest 
of  the  -*■     statements  can  be  obtained  similarly. 

Taking  account  of  all  four  of  I's  (II 's)  strategies,  we  see  that 
I  (II)  can  guarantee  that  the  payoff  will  be  at  least  (at  most) 
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ft1  (a)  +  0(l/n)   and  ftXI(a)  +  0(l/n)  ,   where 

ft  (a)  =  max {min (v   (a) ,a   ,a   ) ,min(v  (a)  ,a   ,a   )  ,V} 

CC      1Z   Zl        c      11  11     — 

ft   (cr)  =  min{max(v   (a)  ,a   ,a   )  ,max{v  (a)  ,a_  0  ,a01  }  ,V} 

cc     11   Z/       c     1Z   Zl 

I       II 
The  fact  that  9.    (a)  =  ft   (a)  =  ft(a)   can  be  shown  by  exhaustion  of  cases. 

Since  ft  (a)  ^  V  and  ft  X(a)  ^  V,   the  proof  is  trivial  if  V  =  V.   If 

there  is  no  saddle  point,  suppose  a    or  a?_   is  the  largest  element 

M.   It  follows  that  the  term  involving  v   (a)   may  be  deleted  from  both 

ft  (a)   and  ft   (a) ,   that  the  second  largest  element  V   is  on  the  same 

diagonal,  and  also  that  v(cr)  =  v  (a).   The  result  is  that 

ft1 (a)  =  max{min{v(a) ,V},V} 

ft11  (a)  =  min{max{v(a),V},V} 

Both  of  these  expressions  are  equivalent  to  1.3.   A  similar  proof  holds 


if   a    or   a    is  the  largest  element. 


QED 
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4 .   Comparison  of  Techniques  and  Prognosis  for  Larger  Games. 

The   2x2   game  would  be  much  easier  to  solve  if  it  could  be 

discovered  that  the  function  ft (a)   satisfies  some  simple  functional 

equation  that  has  a  unique  solution.   This  does  not  seem  to  be  the  case. 

Shubert  deals  with  this  problem  by  inventing  a  situation  that  is  almost 

as  good.   He  introduces  a  function  ft  (a ,X ,k) (x,(A)   in  his  notation) 

which  does  satisfy  such  a  functional  equation,  and  then  shows  that 

lim  ft(a,A,k)  =  fl(a)   regardless  of  the  index  k.   He  next  shows  there 
A+0 

are  only  a  finite  number  of  candidates  for  the  solution  of  the  equation, 

one  of  which  must  apply  for  all  X      smaller  than  some  positive  A~. 

Given  the  matrix  A,   he  finds  the  solution,  and  consequently  ft (a) . 

This  procedure  is  probably  generalizable  to  larger  games.   The  difficulty 

will  be  that  the  number  of  candidate  solutions  will  grow  very  fast  with 

the  size  of  the  matrix.   It  is  possible,  however,  that  a  systematic 

procedure  can  be  found  to  determine  the  correct  solution  without  an 

exhaustive  search. 

Washburn  uses  a  time-honored  technique  for  solving  two-person 

zero-sum  games;  he  guesses  the  optimal  strategies  and  shows  that  they  are 

in  equilibrium.   The  generalization  of  his  procedure  when  I  has  m 

strategies  and  II  has   n  strategies  would  be 

1)  Let  I  (II)  confine  himself  to  all  functions  mapping   {l,...,n(m)} 
onto   {1 , . . . ,m(n) }.   There  are  m  (n  )   such  strategies. 

2)  For  each  such  strategy,  find  the  asmyptotic  bound  that  it  guarantees 
for  the  player  who  is  using  it  by  writing  a  Linear  Program  with  2MN 
variables. 

3)  Show  that  the  greatest  lower  bound  equals  the  least  upper  bound 
(a  theorem  is  required) . 
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Given  the  theorem  in  3) ,  one  could  find  the  ft-value  of  an  m  x  n  game 
for  one  particular  value  of  a  by  solving  min{m  ,n  }  Linear  Programs 
with  2mn  variables  each. 

It  is  apparent  that  either  method  for  finding  the  fi-value  of  a 
large  game  would  require  a  vast  amount  of  computational  effort,  even  if 
all  the  required  theorems  can  be  proved.   The  central  problem  for  matrix- 
differential  games  would  therefore  seem  to  be  the  invention  of  a  practical 
procedure  for  finding  solutions;  it  would  be  particularly  valuable  if 
the  procedure  were  able  to  find  solutions  for  all  a.   By  "practical," 
we  mean  a  procedure  where  computational  effort  increases  less  than 
exponentially  with  m  and  n. 
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